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This  paper  presents  a  Kalman  filter  based  adaptive  disturbance  accommodating  stochastic  control  scheme  for 
linear  uncertain  systems  to  minimize  the  adverse  effects  of  both  model  uncertainties  and  external  disturbances. 
Instead  of  dealing  with  system  uncertainties  and  external  disturbances  separately,  the  disturbance  accommo¬ 
dating  control  scheme  lumps  the  overall  effects  of  these  errors  in  a  to-be-determined  model-error  vector,  and 
then  utilizes  a  Kalman  filter  in  the  feedback  loop  for  simultaneously  estimating  the  system  states  and  the 
model-error  vector  from  noisy  measurements.  Since  the  model-error  dynamics  is  unknown,  the  process  noise 
covariance  associated  with  the  model-error  dynamics  is  used  to  empirically  tune  the  Kalman  filter  to  yield 
accurate  estimates.  A  rigorous  stochastic  stability  analysis  reveals  a  lower  bound  requirement  on  the  assumed 
system  process  noise  covariance  to  ensure  the  stability  of  the  controlled  system  when  the  nominal  control  action 
on  the  true  plant  is  unstable.  An  adaptive  law  is  synthesized  for  the  selection  of  stabilizing  system  process 
noise  covariance.  Simulation  results  are  presented  where  the  proposed  control  scheme  is  implemented  on  a  two 
degree-of- freedom  helicopter. 
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1  Introduction 

Uncertainty  in  dynamic  systems  may  take  numerous  forms,  but  among  them  the  most  significant 
are  noise/disturbance  uncertainty  and  model/parameter  uncertainty.  External  disturbances  and 
system  uncertainties  can  obscure  the  development  of  a  viable  control  law.  This  paper  presents 
the  formulation  and  analysis  of  a  stochastic  robust  control  scheme  known  as  the  Disturbance 
Accommodating  Control.  The  main  objective  of  Disturbance  Accommodating  Control  (DAC)  is 
to  make  necessary  corrections  to  the  nominal  control  input  to  accommodate  for  external  distur¬ 
bances  and  system  uncertainties.  Instead  of  dealing  with  system  uncertainties  and  disturbances 
separately,  DAC  lumps  the  overall  effects  of  these  errors  in  a  to-be-determined  term  that  is  used 
to  directly  update  a  nominal  control  input. 

Disturbance  accommodating  control  was  first  proposed  by  Johnson  in  1971  (Johnson  1971). 
Though  the  traditional  DAC  approach  only  considers  disturbance  functions  which  exhibit  wave¬ 
form  patterns  over  short  intervals  of  time  (Johnson  and  Kelly  1981),  a  more  general  formulation 
of  DAC  can  accommodate  the  simultaneous  presence  of  both  “noise”  type  disturbances  and 
“waveform  structured”  disturbances  (Johnson  1984,  1985).  The  disturbance  accommodating 
observer  approach  has  shown  to  be  extremely  effective  for  disturbance  attenuation  (Biglari  and 
Mobasher  2000,  Profeta  et  al.  1990,  Kim  and  Oh  1998);  however,  the  performance  of  the  observer 
can  significantly  vary  for  different  types  of  exogenous  disturbances,  which  is  due  to  observer  gain 
sensitivity. 
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This  paper  presents  a  robust  control  approach  based  on  a  significant  extension  of  the  con¬ 
ventional  observer-based  DAC  concept,  which  compensates  for  both  unknown  model  parameter 
uncertainties  and  external  disturbances  by  estimating  a  model-error  vector  (throughout  this 
paper  the  phrase  “disturbance  term”  will  be  used  to  refer  to  this  quantity)  in  real  time.  The 
estimated  model-error  vector  is  further  used  as  a  signal  synthesis  adaptive  correction  to  the  nom¬ 
inal  control  input  to  achieve  maximum  performance.  This  control  approach  utilizes  a  Kalman 
filter  in  the  feedback  loop  for  simultaneously  estimating  the  system  states  and  the  disturbance 
term  from  measurements  (Sorrells  1982,  1989,  Davari  and  Chandramohan  2003).  The  estimated 
states  are  then  used  to  develop  a  nominal  control  law  while  the  estimated  disturbance  term  is 
used  to  make  necessary  corrections  to  the  nominal  control  input  to  minimize  the  effects  of  both 
system  uncertainties  and  the  external  disturbance.  There  are  several  advantages  of  implementing 
the  Kalman  filter  in  the  DAC  approach:  i)  tuning  of  the  estimator  parameters,  such  as  the  process 
noise  matrix,  can  be  done  easily  unlike  conventional  DAC  approaches  in  which  the  adaptation 
involves  the  entire  feedback  gain,  ii)  the  estimated  disturbance  term  is  a  natural  byproduct  of 
state  estimation,  and  iii)  the  Kalman  filter  can  be  used  to  filter  noisy  measurements. 

It  is  a  well-known  fact  that  the  closed-loop  performance  and  the  stability  of  the  Kalman 
filter-based  DAC  approach  depends  on  the  accuracy  of  the  estimated  disturbance  term.  Since 
the  dynamics  of  the  disturbance  term  is  unknown,  the  process  noise  covariance  associated  with 
the  disturbance  term  is  used  to  empirically  tune  the  Kalman  filter  to  yield  accurate  estimates. 
Although  the  Kalman  filter-based  DAC  approach  has  been  successfully  utilized  for  practical 
applications,  there  has  not  been  any  rigorous  stochastic  stability  analysis  to  reveal  the  interde¬ 
pendency  between  the  estimator  process  noise  covariance  and  controlled  system  stability.  The 
first  main  contribution  of  this  paper  is  a  detailed  stability  analysis,  which  examines  the  explicit 
dependency  of  the  controlled  system’s  closed-loop  stability  on  the  disturbance  term  process 
noise  covariance  and  the  measurement  noise  covariance.  Since  the  system  under  consideration  is 
stochastic  in  nature,  the  notion  of  stability  is  depicted  in  two  separate  fashions.  The  first  method 
deals  with  moment  stability  and  the  second  technique  considers  stability  in  a  probabilistic  sense. 

Stochastic  stability  analysis  on  the  Kalman  filter-based  DAC  approach  indicates  that  the 
effectiveness  of  the  proposed  control  scheme  depends  on  the  estimator  parameters  such  as  the 
process  noise  covariance  matrix.  The  stability  analysis  also  indicates  that  the  DAC  scheme  is 
most  effective  when  the  assumed  process  noise  covariance  satisfies  a  lower  bound  requirement 
which  depends  on  the  system  uncertainties.  In  general,  it  is  difficult  to  select  a  stabilizing  process 
noise  covariance  for  the  broad  type  of  uncertain  systems  considered  here.  One  could  always  try 
to  select  an  extremely  large  value  of  process  noise  covariance  that  might  stabilize  the  system 
or  even  monotonically  increase  the  process  noise  covariance  matrix  in  an  ad-hoc  manner  until 
the  system  stabilizes.  However,  it  is  important  to  keep  in  mind  that  selecting  a  large  process 
noise  covariance  matrix  would  result  in  noisy  control  signal  which  could  lead  to  problems  such  as 
chattering.  The  second  main  contribution  of  this  paper  is  the  formulation  of  a  stochastic  adaptive 
scheme  for  selecting  the  appropriate  process  noise  covariance  that  would  guarantee  closed-loop 
stability  of  the  controlled  system. 

The  structure  of  this  paper  is  as  follows.  A  detailed  formulation  of  the  stochastic  DAC  approach 
for  multi-input  multi-output  (MIMO)  systems,  followed  by  a  stochastic  stability  analysis,  is  first 
given.  Next,  an  adaptive  scheme  is  developed  for  the  selection  of  stabilizing  the  disturbance 
term  process  noise  covariance.  Simulation  results  are  then  presented  where  the  proposed  control 
scheme  is  implemented  on  a  two  degree-of-freedom  helicopter. 


2  Disturbance  Accommodating  Controller  Formulation 

Let  (ft,  T ,  P)  denote  a  probability  space,  where  ft  is  the  sample  space,  J7  is  a  cx-field,  and  P  is  a 
probability  measure  on  the  measurable  space  (ft,  J-).  Additionally,  the  elements  of  ft  are  denoted 
by  oj  and  the  members  of  T  are  called  events.  Now  consider  a  linear  time-invariant  stochastic 
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system  of  the  following  form: 

X1(i)  =  A1X1(t)  +  A2X2(i),  X1(to)  =  Xlo  1 

X2(t)  =  AaX^t)  +  A4X2(t)  +  Bu(t)  +  W(t),  X2(t0)  =  X2o 

Here,  the  stochastic  state  vector,  [X^(t)  X^(i)]7  =  X(t)  =  X(t,ca)  :  [toU/]  x  H  i-a  51in,  is  an 
n-dimensional  random  variable  for  fixed  t.  The  state  vectors,  Xi  (t)  and  X2(t)  are  of  dimensions 
Xi (t)  =  Xi (t,u)  :  [to,tf]  x  H  i-a  3in_r  and  X2(t)  =  X2(i,u;)  :  [to,tf]  xSln  ;Rr,  respectively. 
The  system  given  in  (1)  is  in  the  typical  kinematics-dynamics  form,  where  the  kinematics  is 
assumed  to  be  fully  known,  i.e.,  the  state  matrices  A\  £  s jj(n-r)  x  (n-r)  an(]  A2  g  sjj(n-r)xr  are 
precisely  known.  Uncertainty  is  only  associated  with  the  dynamics,  i.e.,  the  state  and  control 
distribution  matrices,  A3  £  ^Rrx^n~r\  A4  £  Wxr,B  £  3?rxr,  are  assumed  to  be  unknown.  Also, 
the  input  matrix,  B  is  assumed  to  be  nonsingular.  Finally,  the  stochastic  external  disturbance 
W(t)  =  W :  [to,tf]  x  1— >■  !Rr  is  modeled  as  a  linear  time-invariant  system  driven  by  a 
Gaussian  white  noise  process,  i.e., 

W(t)  =L(X(t),W(t))  +  V(*),  W(t0)  =  O,.xl  (2) 

where  L(-)  is  an  unknown  linear  operator  and  V(t)  =  V(t,oj)  :  xHa  3?r,  is  assumed  to 

be  zero-mean  Gaussian  white  noise  process,  i.e.,  V(t)  ~  N(0,  Q<5(r)).  It  is  important  to  note 
that  the  linear  operator  L(-)  and  the  covariance  of  the  white  noise  process  V(i),  are  unknown. 
The  measurement  equation  is  given  as 

Y  (t)  =  CX(t)  +  V  (t)  (3) 

where  Y (t)  =  Y (t,u)  :  [to,tf]  xfle>  is  the  measurement  vector  and  C  £  sjjmxn  denotes  the 
known  output  matrix.  The  measurement  noise,  V(t)  =  V(t,  w)  :  [to,tf]  xHa  is  assumed 
to  be  zero-mean  Gaussian  white  noise  with  known  covariance,  i.e.,  V(t)  ~  M(0,  RS(t)^  . 

The  assumed  (known)  system  matrices  are  given  as  A3m,  A4m,  and  Bm.  Now  the  external 
disturbance  and  the  model  uncertainties  can  be  lumped  into  a  disturbance  term,  'D(t)  £  3?r, 
through 


V(t)  =  AAi~X.i(t)  +  AA2X2(f)  +  ABu(f)  +  W(t)  (4) 

where  AAi  =  (A3  —  A3m),  AA2  =  {A4  —  A4m)  and  A B  =  (B  —  Bm).  Using  this  disturbance  term 
the  true  model  can  be  written  in  terms  of  the  known  system  matrices  as  follows: 

Xi(t)  =  AiX^t)  +  A2X2(t) 

(5) 

x2(t)  —  A3mXi(t)  +  A4mX2(i)  +  Bmu(t)  +  T>(t) 

The  control  law,  u (t),  consists  of  a  nominal  control  and  a  control  correction  term  to  minimize 
the  adverse  effect  of  the  disturbance  term,  i.e., 

u  (t)  =  u(t)  +  u  (t)  (6) 

Here  u (t)  is  the  nominal  control  and  u(t)  is  the  control  correction  term.  For  the  purpose  of  anal¬ 
ysis,  the  control  correction  term  is  selected  to  ensure  the  complete  cancelation  of  the  disturbance 
term.  Thus  the  disturbance  accommodating  control  law  can  be  written  as 


u(t)  =  u(t)  - 


(7) 
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The  disturbance  term  is  not  known,  but  an  estimator  can  be  implemented  in  the  feedback  loop 
to  estimate  the  disturbance  term  online.  Estimating  the  disturbance  term  requires  knowledge 
of  its  dynamic  model.  Since  the  dynamics  of  the  disturbance  term  is  not  precisely  known,  the 
disturbance  term  dynamics  is  modeled  as 

=  Ax>m'Dm(t )  +  W(t),  T>m(to)  =  0  (8) 

where  Ax>m  is  Hurwitz  and  W(t)  =  W :  [to,tf]  x  fl  i-a  3ir  is  zero-mean  Gaussian  white 
noise  process,  i.e.,  W  (t)  A/"(0,  Q5(t))  .  Equation  (8)  is  used  solely  in  the  estimator  design  to 
estimate  the  true  disturbance  term.  After  constructing  the  assumed  augmented  state  vector  as 
Z m(t)  =  [X^  (t)  (t)  T>Jn(t)]  ,  the  assumed  extended  model  of  the  system  can  be  written  as 


xi  m(t,y 
X2m(i) 

— 

A2  0  (n—r)xr 
A^  m  ^4  m  I rxr 

"Xim(*)' 

X2m(i) 

+ 

1 

X 

T  £ 

^cq 

O" 

1 _ 

U  (f)  + 

H(n— r)  x  1 
Orxl 

(9) 

0 rx(n—r )  ^rxr 

Orxr 

.  W(f)  _ 

The  zero  elements  in  the  disturbance  term  dynamics  are  assumed  for  sake  of  simplicity,  however 
the  control  formulation  given  here  is  also  valid  if  nonzero  elements  are  assumed.  Equation  (9) 
can  be  written  in  terms  of  the  appended  state  vector,  Zm,  as 

z m(t)  =  FmZm(t)  +  Dmu(t)  +  GW(t),  Zm{t0)  =  [Xf0  x£  0r]T  (10) 

where 


F  — 

1  m  — 

A\ 

^3m 

A2  0(n_r)xr 
A.4m  Irxr 

■>  Dm  — 

1 

X 

T  £ 

I05 

0 

1 _ 

,  G  = 

0  nxr 
I rxr 

_^rx(n—r) 

0  rxr  A x>m 

Orxr 

Note  that  the  uncertainty  is  only  associated  with  the  dynamics  of  the  disturbance  term.  Let 
Z(t)  =  [Xf  (t)  X^f)  T>t (t)\  and  H  =  [C  0mxr].  Now  the  measured  output  equation  can  be 
written  as 


Y  (t)  =  HZ{t)  +  V  (t) 


(11) 


Though  the  disturbance  term  is  unknown,  an  estimator  such  as  a  Kalman  filter  can  be  imple¬ 
mented  in  the  feedback  loop  to  estimate  the  unmeasured  system  states  and  the  disturbance  term 


from  the  noisy  measurements.  Let  Z(t)  = 
can  be  written  as 


Xj(t)X^(t)T>  ( t ) 


lT 


now  the  estimator  dynamics 


Z (t)  =  FmZ(t)  +  Dmu(t)  +  K{t)[Y(t)  -  Y(t)],  Z(t0)  =  Z m(t0)  (12) 

where  K(t)  is  the  Kalman  gain  and  Y (t)  =  FLZ(t).  The  Kalman  gain  can  be  calculated  as 
K{t )  =  P(t)HT i?_1,  where  P{t)  is  obtained  by  solving  the  continuous-time  matrix  differential 
Riccati  equation  (Crassidis  and  Junkins  2004): 

P(t)  =  FmP(t)  +  P(t)FmT  -  P(t)HT R~x HP(t)  +  GQGt,  P(to)  (13) 

Let  Z (t)  =  [Xj(t)  XZ(t)  T>T(t)]7  ,  now  the  estimator  dynamics  can  be  rewritten  as 
Z (t)  =  FmZ(t)  +  Dmu(t)  +  K(t)H[Z(t)  -  Z{t)\  +  K(t)V(t ) 


(14) 
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The  estimator  uses  the  assumed  system  model  given  in  (10)  for  the  propagation  stage  and  the 
actual  measurements  for  the  update  stage,  i.e.,  Z (t)  =  E[Zm(t)\{Y  (to) . . .  Y(t)}].  Due  to  system 
uncertainties,  the  estimator  in  (14)  is  sub-optimal  and  the  estimates  Z  (t)  may  be  biased. 

Remark  1 :  Accuracy  of  the  estimates  depends  on  Q  which  indicates  how  well  the  disturbance 
term  dynamics  is  modeled  via  (8).  A  large  Q  indicates  that  (8)  is  a  poor  model  of  the  true 
disturbance  term  dynamics  and  a  small  Q  indicates  that  (8)  is  an  accurate  model  of  the  true 
disturbance  term  dynamics.  Note  that  selecting  a  small  Q ,  while  having  a  poor  model,  would 
results  in  inaccurate  estimates. 

The  total  control  law,  u(t),  consists  of  a  nominal  control  and  necessary  corrections  to  the 
nominal  control  to  compensate  for  the  disturbance  term  as  shown  in  (7).  The  nominal  control  is 
assumed  to  be  a  state  feedback  control,  where  the  feedback  gain,  Km  =  [ Kmi  Km.2 ] ,  is  selected 
so  that  (Am  —  BmKm)  is  Hurwitz,  where 


A 


m 


A\  A2 
A3m  Aim 


and  Bm 


0 


(n-r)xr 

Bm 


While  the  nominal  controller  guarantees  the  desired  performance  of  the  assumed  model,  the 
second  term,  — X>(t),  in  (7)  ensures  the  complete  cancelation  of  the  disturbance  term  which 
is  compensating  for  the  external  disturbance  and  model  uncertainties.  Now  the  disturbance 
accommodating  control  law  can  be  written  in  terms  of  the  estimated  system  states  and  the 
estimated  disturbance  term  as 


u(t) 


Km  B 


-1 


-±(t) 

P(t)_ 


SZ(t) 


(15) 


where  S  = 


B 


-l 


.  Notice  that  Bm 


is  assumed  to  be  a  nonsingular  matrix.  A  summary  of 


the  proposed  control  scheme  is  given  in  Table  1. 


Table  1.  Summary  of  Disturbance  Accommodating  Control  Process 


Plant 

Xi(t)  =  A1X1(t)  +  A2X2(t) 

X2  (t)  =  A3mXi(t)  +  A4mX2(t)  +  Bmu(t)  +  X>(t) 

Y(t )  =  CX(t)  +  V(t) 

Initialize 

Z(to),  P(to) 

Estimator  Gain 

P(t)  =  Fmp(t )  +  P(t)FmT  -  P(t)HTR~1HP(t)  +  GQGt 

K(t )  =  P(t)HTR~l 

Estimate 

Z(i)  =  FmZ(t)  +  Dmu(t)  +  K (t )  \Y(t)  -  Y(t)l 

Control  Synthesis 

u (t)  =  -  Km  Bml  Z(t) 

Remark  2  :  It  is  important  to  note  that  if  Q  ~  0,  then  T>m(t)  ~  T>m,(to )  =  0,  and  the  total 
control  law  given  in  (15)  becomes  just  the  nominal  control.  If  the  nominal  control,  u(f),  on  the 
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true  plant  would  result  in  an  unstable  system,  i.e. ,  the  matrix  (A  —  BKm )  is  unstable,  where 


Ai  A2 

and  B  = 

a 

X 

"5 

- 1 

A3  Ap 

L  *  \ 

then  selecting  a  small  Q  would  also  result  in  an  unstable  system.  On  the  other  hand,  selecting 
a  large  Q  value  would  compel  the  estimator  to  completely  rely  upon  the  measurement  signal 
and  therefore  the  noise  associated  with  the  measurement  signal  is  directly  transmitted  into 
the  estimates.  This  could  result  in  a  noisy  control  signal  which  could  lead  to  problems  such 
as  chattering.  Also  note  that  as  R,  the  measurement  noise  covariance,  increases,  the  estimator 
gain  decreases  and  thus  the  estimator  fails  to  update  the  propagated  disturbance  term  based  on 
measurements.  Thus,  for  a  highly  uncertain  system,  if  the  nominal  control  action  on  the  true 
plant  would  result  in  an  unstable  system,  then  selecting  a  small  Q  or  a  large  R  would  also  result 
in  an  unstable  closed-loop  system. 

If  the  estimator  in  (14)  is  able  to  obtain  accurate  estimates  of  the  system  states  and  the 
disturbance  term,  then  the  control  law  in  (15)  guarantees  the  desired  closed-loop  performance. 
The  accuracy  of  the  estimated  system  states  and  the  disturbance  term  depends  on  the  estimator 
parameters  such  as  the  process  noise  covariance,  Q,  and  the  measurement  noise  covariance,  R. 
Thus  the  performance  of  the  DAC  approach  presented  here  depends  on  the  estimator  design 
parameters.  A  schematic  representation  of  the  proposed  controller  is  given  in  figure.  1. 


Figure  1.  DAC  Block  Diagram 


3  Stochastic  Stability  Analysis 

Without  loss  of  generality,  the  following  assumption  can  be  made  about  the  external  disturbance 
model. 

Assumption  3.1  :  The  linear  operator,  L(-),  in  the  external  disturbance  term  model  in  (2)  is 
assumed  to  be 


L(X(t),W(t))  =  yl_wiXi(t)  +  ^4W2X 2(1)  H-  ^4w3  W(t) 


(16) 


where  AWl,  AW2,  and  AW3  are  unknown  matrices. 

Based  on  equation  (4),  the  true  disturbance  term  dynamics  can  now  be  written  as 


±>(t)  =  AAiXi(t)  +  A  A2X2(t)  +  ABu(t)  +  L  (X(t),  W(t))  +  V(t) 


(17) 
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Substituting  the  control  law  (15)  the  above  equation  can  be  written  as 

±>(t)  =AA1K1(t)  +  A  A2X2(t)  +  ABSZ(t)  +  L  (X(i),  W(i))  +  V(t) 

=AAX  {A&iit)  +  A2X2(f)}  +  AA2  {A3mX!(t)  +  A4mX2(f)  +  BmSZ(t )  +  Z>(t)}  + 
ABS  {[Fm  +  BmS  -  X(t)B]  Z(t)  +  K{t)HZ(t)  +  X(t)V(f)}  + 

AWlXi(f)  +  AW2X2(f)  +  AW3  W  (t)  +  V(t) 

Assume  the  output  matrix  can  be  partitioned  as  C  =  [C\  C2\ ,  and  H  can  be  written  as  H  A 
[Cl  C2  0mxr].  Thus  K(t)HZ(t)  =  K(t)Ci^i{t)  +  K(t)C2X.2(t).  Also  note  that  W (t)  can  be 
written  as  W (t)  =  T>(t)  —  AA4Xi(t)  —  AA2X2(t)  —  A5u(t).  Now  the  true  disturbance  term 
dynamics  can  be  written  as 

±>{t)  =  {AA\Ai  +  AA2A3m  +  ABSK{t)Ci  +  AWl  -  AW3AA4}  X4(t)+ 

{  AAiA2  +  AA2A4m  +  A BSK(t)C2  +  AW2  -  AW3AA2}  X2(t)+ 

{ABS  [Fm  +  DmS  -  K(t)H\  +  A A2BmS  -  AW3ABS}  Z (t)+ 

{AA2  +  AW3}  V(t)  +  A  BSK(t)V(t)  +  V(t) 

Let  AB S K (t)Y (t)  +  V(i)  =  W a(t),  thus  Wa(t)  is  also  a  zero-mean  stochastic  process  with 

E  [Wa(t)WZ(t  +  r)]  =  {ABSK{t)RKT(t)STABT  +  Q)  5{t)  =  Qa{t)5{T) 

Notice  T>(t)  is  a  linear  function  of  the  true  extended  system  state,  Z (t),  the  estimated  states, 
Z (t),  and  the  noise  term,  W a(t).  Thus  the  system  in  (1)  is  rewritten  as  the  following  extended 
dynamically  equivalent  system: 


'Xi(t)' 

A, 

a2 

0(n— r)  xr 

'Xi  (t) 

b(n— r)  X  (n+r) 

0(n— r)xl 

X2(t) 

= 

A3m 

A4m 

Irxr 

X2(t) 

+ 

BmS 

Z(t)  + 

Orxl 

(18) 

V(t). 

\_Av1  (t)  AV2  (t) 

av3  (t)  _ 

V(t) 

Bv(t ) 

.  Wa(i)  _ 

where 

dpi  (t)  =  {AA1A1  +  AA2A3m  +  ABSK(t)C\  +  AWl  —  AW3AAi| 
Ax>2  (t)  =  {AAiA-2  +  A  A2A4m  +  A  BSK(t)C2  +  AW2  —  AW3AA2} 
Avs(t)  =  {A  A-2  +  ^W3} 

and 


Bv(t)  =  { ABS  [Fm  +  DmS  -  K(t)H]  +  A A2BmS  -  AW3ABS} 

Equation  (18)  can  be  written  in  concise  form  as 


Z(t)  =  F{t)Z(t)  +  D(t)Z(t)  +  GWa(t) 


(19) 
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where 


A\ 

A  2 

0(n— r)  xr 

0(n— r)  X  (n+r) 

II 

ST 

A3m 

A-4m 

Irxr 

II 

^+o' 

£ 

BmS 

Ax>2{t) 

A 

CO 

1 _ 

Bv(t) 

After  substituting  the  control  law,  the  estimator  dynamics  can  be  written  as 
Z(t)  =  FmZ{t)  +  DmSZ(t )  +  K(t)H[Z(t)  -  Z (t)\  +  K(t)V(t) 


Let  Z(t)  =  Z (t)  —  Z(t)  be  the  estimation  error,  then  the  error  dynamics  can  be  written  as 

Z(f)  =  [Fm  -  I<(t)H  +  AF{t)]Z{t)  +  [A F(t)  +  AD(t)]Z{t)  +  GWa(t)  -  K(t)V(t)  (20) 

where  A F(t)  =  F(t)  —  Fm  and  A D(t)  =  D(t)  —  DmS.  Combining  the  error  dynamics  and  the 
estimator  dynamics  yields 


i 

-+o 

•<N 

1 - 

\Fm  -  K(t)H  +  A F(t))  (AF(t)  +  A D(t)) 

Z(t) 

+ 

i 

'Wa(t) 

(. Fm  +  DmS ) 

m. 

0(n+r)xr  K(£) 

V(t)  _ 

(21) 


or  in  a  more  compact  form  as 


z(t)  =  r(t)z{t)  +  T(t)G{t) 


(22) 


where 


m 


(Fm  -  K(t)H  +  AF(t))  (A F(t)  +  A D(t)) 
K(t)H  (Fm  +  DmS)  ’ 


m 


m 


m 

m 


m 


W0(() 

.  V(t) 


G  -K(t) 

0(n+r)xr  -^(^) 


Although  the  Kalman  filter-based  DAC  approach  has  been  successfully  utilized  for  practical 
applications,  there  has  not  been  any  rigorous  stochastic  stability  analysis  to  reveal  the  interde¬ 
pendency  between  the  estimator  process  noise  covariance  and  controlled  system  stability.  Since 
the  system  under  consideration  is  stochastic  in  nature,  the  notion  of  stability  is  depicted  in  two 
separate  fashions.  The  first  method  deals  with  moment  stability;  for  the  Gaussian  stochastic 
processes  presented  here,  the  first  two  moments  are  considered.  The  second  technique  considers 
stability  in  a  probabilistic  sense. 


3.1  First  Moment  Stability 

In  this  section  a  detailed  stability  analysis  which  examines  the  explicit  dependency  of  the  con¬ 
trolled  system’s  first  moment  stability  or  the  mean  stability  on  the  estimator  parameters,  such 
as  the  disturbance  term  process  noise  covariance  Q,  and  the  measurement  noise  covariance  R,  is 
given.  First,  a  few  definitions  regarding  the  closed-loop  system’s  mean  stability  are  given.  These 
definitions  and  notations  are  first  introduced  for  a  system  without  any  parameter  uncertainties 
and  are  used  throughout  the  rest  of  this  manuscript. 
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3.1.1  System  without  Uncertainties 

Here  a  system  without  any  parameter  uncertainties  is  considered,  i.e. ,  F(t)  =  Fm.  D(t)  = 
DmS,  and  W a(t)  =  W(t).  If  there  is  no  model  error,  then  the  estimator  is  unbiased,  i.e., 

E  Z(t)  =  n^(t)  =  0.  Note  that  the  over  line  is  used  to  indicate  the  states  of  the  system  when 
there  is  no  model  uncertainties.  Now  (21)  may  be  written  as 

Z(t)l  =  \Fm-K(t)H  0(n+r)x(n+r)l  [z(t)l  \  G  -K(t)  1  \W(t) 

L  K(t)H  Fm  +  DmS\  Z  (t)  [°(n+r)xr  ^(0j[V(0. 

where  Z(t)  and  Z  (t)  denote  the  estimation  error  and  estimated  states  when  there  is  no  model 

r_  _  1  T  rp 

error,  respectively.  Let  Z(t)  =  Z T(t)  Z T(t)  and  Q(t)  =  [Wr(t)  VT(t)]^  ,  now  the  above 
equation  can  be  written  in  a  more  compact  form  as 

2(t)  =  ?(t)Z(t)  +  r(t)6(t)  (23) 

where 

_  -Lm  K(t)H  0(n_|_r)x(n_|_r)l 

K(t)H  Fm  +  DmS 

Notice  that  Q(t)  is  a  zero-mean  Gaussian  white  noise  process  with 

E[Q[t)G‘\t  ~  r)]  =  L  Q  °’H  <S(t)  =  A.S(t) 

umxr 

Since  the  first  moment  stability  is  of  concern  here,  the  first  moment  dynamics  or  the  mean 
dynamics  is  written  as 

E[Z(t)}  =  Uz(t)  =  T(t)/**(0  (24) 

Definition  3.2:  Given  M  >  1  and  (3  €  R,  the  system  in  (23)  is  said  to  be  (M,/3)- stable  in  the 
mean  if 

|4>(Mo)/Lz(*o)I  <  Vt  >  t0  (25) 

where  <b(t,io)  is  the  evolution  operator  generated  by  T(£)  and  |  •  |  indicates  the  Euclidean  norm, 
i.e., 

|m|  =  \J m\  +  +  . . . 

Since  most  applications  involve  the  case  where  /3  <  0,  (M ,  /^-stability  guarantees  both  a 
specific  decay  rate  of  the  mean  response  (given  by  /3)  and  a  specific  bound  on  the  transient 
behavior  of  the  mean  (given  by  M). 

Definition  3.3 :  If  the  stochastic  system  in  (23)  is  (M,  /?)-stable  in  the  mean,  then  the  transient 
bound  of  the  system  mean  response  for  the  exponential  rate  /3  is  defined  to  be 

Mp  =  inf | M  <E  R;Vt  >  t0  :||  $(t,t0)  ||<  Mep{t~to)} 


(26) 
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Here  ||  •  ||  indicates  the  matrix  two-norm,  i.e., 

II  M  ||=  crmax(M) 

where  crmax(-)  denotes  the  maximum  singular  value. 

As  shown  in  the  Theorem  given  below,  the  (M,  /3)-stability  and  the  transient  bound  of  the 
system’s  mean  response  are  related  to  a  continuous  time  Lyapunov  matrix  differential  equation. 

Theorem  3.4 :  Assume  there  exists  a  bounded,  continuously  differentiable  positive  definite 
matrix  function  Pit)  satisfying  the  Lyapunov  matrix  differential  equation 

v(t)  =  r(t)P(t)  +  P{t)rT{t)  +  r(t)ArT(f),  P(t0)  (27) 

then  the  system  in  (23)  is  (M,  f3)-stable  in  the  mean  and  the  transient  bound  Mp  of  the  system 
mean  response  can  be  obtained  as 

Mf  <  sup  amxx{V(t))/amm{V{tn))  (28) 

t>to 

where  Uminf-)  denotes  the  minimum  singular  value. 

Proof  Since  r(f)Ar2  ( t. )  >  0  Vf  >  to,  the  (M, /3)-stability  in  the  mean  follows  directly  from  the 
existence  of  bounded  positive  definite  solution,  P(t),  satisfying  equation  (27).  Now  the  solution 
to  (27)  can  be  written  as 

p(t)  =$(t,t0)V(to)$T(t,t0)  +  [  ^(t,r)r(r)ArT(r)^T(t,r)(iT 

Jt0 

Notice  that  Vf  >  t0,  P(t)  >  $(i,  t0)V(t0)$T (t,  t0)  >  amin(P(t0))®(t,  t0)$T(t,  t0),  i.e., 

& . max  (P(t))  >||  $(t,t0)V(t0)$T(t,to)  ||  ^  ^min  ||  ||2,  t  >  t0 

Now  (28)  follows  from 

^"max  (Pit))/ 

^min  (V(t0))  >||  4>(Mo)  ||2,  t  >  to 

□ 

_  —  —  —  rji 

Remark  3:  Assume  V(to )  is  selected  as  V(to)  =  E[Z{to)Z  (to)];  then  the  positive  definite 
solution,  V(t),  satisfying  equation  (27)  denotes  the  correlation  matrix,  i.e., 

Pit)  =  E[Z(t)ZT(t )] 

Thus  the  transient  bound  of  the  system  mean  response  can  be  obtained  in  terms  of  the  bounded 
correlation  matrix. 

Note  that  T[t)ATT (t)  in  (27)  can  be  factored  as  shown  below: 

,  =  ,  _  \(GQGt  +  KRKt)  -KRKt]  _  \GQGt  -0]  [  KRKt  -KRKt~ 

1  1  '  1  1  “  [  -KRKt  KRKt  J  “  [  0  0  J  +  [-ERE1  ERET 

=  q  Q[GT  0]  +  Pp^T  R~l  [HP  -HP]  =  LQLt  +  N(t)R~1NT(t) 
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where 


L  = 


G 

0 


and  N  (t) 


'  P{t)HT  ' 
—P(t)HT 


Thus  (27)  can  be  written  as 

V(t)  =  T  (t)P(t)  +  P{t)fT(t)  +  LQLt  +  N(t)R~1NT(t)  (29) 

3.1.2  Uncertain  System 

In  this  subsection,  the  first  moment  stability  of  the  perturbed  system  given  in  (22)  is  consid¬ 
ered,  i.e. , 


Z(t)  =  T  (t)Z(t)  +  AT  (t)Z{t)  +  T(t)Q{t) 


(30) 


where 


AT(t) 


A F{t)  (A F(t)  +  A D(t)j 
0  0 


The  correlation  matrix  V{t)  =  E  [ Z(t.)Z 1  (t)]  satisfies  the  following  matrix  Lyapunov  differential 
equation: 


V(t)  =  (T(t)  +  AT(t))  V(t)  +  V(t)  (T (t)  +  AT (t)f  +  T(t)A(t)TT(t)  (31) 


where 


A (t)S(r)  =  E[G(t)GJ  (t  -  r)] 


Qa(t)  0  rxm 

0  mxr  R 


5(t) 


Assuming  the  nominal  control  action  on  the  true  plant  would  result  in  an  unstable  system,  sta¬ 
bility  of  extended  uncertain  system  given  in  (30)  depends  on  the  disturbance  term  process  noise 
covariance,  Q,  and  the  measurement  noise  covariance,  R.  The  Theorem  given  below  indicates 
that  the  stability  of  the  extended  uncertain  system  given  in  (30)  is  guaranteed  if  the  selected  Q 
and  R  satisfies  a  lower  and  an  upper  bound,  respectively. 

Theorem  3.5:  The  uncertain  system  in  (30)  is  (M,  fT) -stable  in  the  mean  if 

Klin (Q)  +  VminiR-1)  ||  A^A^t)  ||  -l}  ||  V(t)  H’2  >  2  ||  AT(t)  ||2,  t  >  t0  (32) 
where  V(t)  satisfies  the  matrix  differential  equation 
■p(t)  =  T  (t)V(t)  +  V{t)fT(t)+  ||  Q  ||  /+  ||  NftfR-'Nftf  ||  I  -  V{t)ATT(t)AT{t)V{t)  (33) 


Proof  For  the  linear  time- varying  system  given  in  (30),  uniform  asymptotic  stability  in  the  mean 
implies  (M,  /5)-stability  in  the  mean.  In  order  to  show  the  uniform  asymptotic  stability  of  the 
mean,  consider  the  mean  dynamics  of  the  system  in  (30): 


=  T  {t)nz{t)  +  AT  (t)nz(t) 


(34) 
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where  E  [Z{t)\  =  nz{t).  Construct  the  following  Lyapunov  candidate  function: 

V\Vz{t)\  =  (35) 

Note  that  the  solution,  P(t),  of  (33)  is  required  to  be  a  bounded  positive  definite  matrix  as  long 
as  AT  is  norm-bounded  (Abou-Kandil  et  al.  2003).  Thus  P~l(t)  exists  and  V[/J,z(t)\  >  0  for  all 
Hz{t)  p  0.  Since  P(t)P~l(t)  =  /,  the  time  derivative  of  P(t)P~l{t)  is  0: 

jt  P^P^it)  =  P(t)P~1(t)  +  P{t)P~1{t)  =  0 

Solving  the  above  equation  for  P~l(t)  and  substituting  (33)  gives 

p~\t)  =  -p-\t)P(t)p-\t) 

=  -P~l{t)T{t)  -  rT(t)p-\t)  -  {  ||  Q  ||  +  ||  N(t)R~1N(t)7  ||  }p-l(t)P~l(t) 

+  AT7{t)AT{t) 

Now  the  time  derivative  of  (35)  can  be  written  as 

V[nz{t)\  =LLzP~lVz  +  SzP-'hz  +  Hz,P~lPz 

=[T  nz  +  AT  fizfp-^z  -  ^tzP-1T^z  - 

{  ||  Q  ||  +  ||  NR^N7  ||  }iiTzP~2Hz  +  /x|ATtAT hz  +  VzP^ftnz  +  AT [iz\ 
=filArTp-1fiz  +  RzV~1ATfiz  -  {  ||  Q  ||  +  ||  NR-'N7  ||  }^TzP~2tiz 

+  tilAT7  AT  tiz 

=fi^^AT7P~1  +  p-1  AT  -  {  ||  Q  ||  +  ||  NR-'N7  ||  }P~2  +  AT7 AT  jfiz 

Asymptotic  stability  in  the  first  moment  is  guaranteed  if 

{AT^-1  +  p-1  AT  -  {  ||  Q  ||  +  ||  NR-'N7  ||  }P~ 2  +  ATTAT}  <  0 

Note 

r  ^ir  w  I  t  „  „  „ 

AT7  —  "p- 1  ATt  -  V~l  >  0  =>  ATtAT  +  V~2  >  AT^”1  +-p_1AT 

Thus  the  above  condition  for  asymptotic  stability  is  satisfied  as  soon  as 

|2ATtAT  +  P~2  -  {  ||  Q  ||  +  ||  NR^N7  ||  }P"2}  <  0 
or 

1 2pAT7ATp  + 1  -  {  ||  Q  ||  +  ||  NR_1N7  ||  }/}  <  0 
Using  the  inequalities 

||  P  ||2||  AT  ||2  /  >  pAT7ATp,  amin(Q)  <||  Q  ||,  and  cx^iT1)  ||  NN7  ||<||  NR^N7  || 
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yields 


2  ||  V  ||2||  AT  ||2<  <7min(Q)  +  <Tmin(i?_1)  ||  NNt  ||  -1 
Hence  the  uniform  asymptotic  stability  in  the  first  moment  is  guaranteed  if 

2  ||  AT(t)  ||2<  {amin(Q)  +  <Tmin(i?-1)  ||  N{t)NT{t)  ||  -1}  ||  V(t)  H"2,  t  >  t0 


□ 

Remark  4  :  The  uncertain  system  in  (30)  is  (M, /?)- stable  in  the  mean  if  the  selected  Q  and  R 
satisfy  the  inequality  in  (32).  Thus  for  a  highly  uncertain  system,  if  the  nominal  control  action 
on  the  true  plant  would  result  in  an  unstable  system,  then  selecting  a  small  Q  or  a  large  R  would 
also  result  in  an  unstable  closed-loop  system. 


3.2  Mean  Square  Stability 

In  this  subsection  the  controlled  system’s  stability  in  the  second  moment  or  the  mean  square 
stability  is  considered.  It  is  shown  here  that  the  (M,  /^-stability  in  the  mean  implies  mean 
square  stability.  More  details  on  mean  square  stability  can  be  found  in  Kushner  (1967)  and 
Soong  (1973). 

Definition  3.6:  A  stochastic  system  of  the  following  form  Z(t)  =  T {t)Z(t)  +  T{t)G(t)  is  mean 
square  stable  if 

lirn  E[ZT{t)Z{t)}  <  M  (36) 


where  M  is  a  constant  square  matrix  whose  elements  are  finite. 

Note  that  E[ZT(t)Z(t)\  =  Tr{T(t )},  i.e., 

jtE[z{t)zT(t)\  =  vit)  =  r(t)v(t)  +  v(t)rT{t)  +  r(t)A{t)rT(t) 

and  the  solution  to  the  above  equation  can  be  written  as 

V(t)  =  (  4>(f,  r)r(r)A(r)ri  (r)4>T(t,  r)dr 

J  —oo 


The  (M,  /3)-stable  in  the  mean  implies  the  system  matrix,  T(t)  =  T(i)  +  AT(t),  generates  an 
exponentially  stable  evolution  operator,  and  therefore  V(t)  has  a  bounded  solution  (Abou-Kandil 
et  al.  2003).  Therefore,  for  the  system  given  in  (30),  (M ,  (0)-stability  in  the  mean  implies  mean 
square  stability. 


3.3  Almost  Sure  Asymptotic  Stability 

The  solution  to  the  stochastic  system  given  in  (30)  cannot  be  based  on  the  ordinary  mean  square 
calculus  because  the  integral  involved  in  the  solution  depends  on  G(t),  which  is  of  unbounded 
variation  (Soong  and  Grigoriu  1993).  For  the  treatment  of  this  class  of  problems,  the  stochastic 
differential  equation  may  be  rewritten  in  Ito  form  as 


dZ(t)  =  [T (t)Z(t)  +  AT(t)Z(t)]  dt  +  F(t)A1/2(t)dB(t) 
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or  simply  as 


dZ(t)  =  T  (t)Z(t)dt  +  T{t)A1/2{t)dB{t)  (37) 

where  dB(t)  is  an  increment  of  Brownian  motion  process  with  zero- mean,  Gaussian  distribution 
and 


E[dB(t)dBr  {t)]  =  Idt  (38) 

The  solution  Z(t)  of  (37)  is  a  semimartingale  process  that  is  also  a  Markov  process  (Grigoriu 
2002).  Details  on  the  almost  sure  ( a.s .)  stability  for  the  stochastic  system  in  (37)  is  presented 
in  this  section. 

Definition  3.7:  The  linear  stochastic  system  given  in  (37)  is  asymptotically  stable  with 
probability  1,  or  almost  surely  asymptotically  stable,  if 

P (Z(t)  — >  0  as  t  — >  oo)  =  1  (39) 

(M,  /3)-stability  in  the  mean  response  implies  that  T  (f)  generates  an  asymptotically  stable 
evolution  for  the  linear  system  in  (37),  but  it  does  not  imply  almost  sure  asymptotic  stability 
due  to  the  persistently  acting  disturbance.  In  fact,  given  Y(t)  generates  an  asymptotically  stable 
evolution,  the  necessary  and  sufficent  condition  for  almost  sure  asymptotic  stability  is 

lim  ||  T(f)  ||2  log(f)  =  0  (40) 


A  detailed  proof  of  this  argument  can  be  found  in  Appleby  (2002).  Equation  (40)  constitutes  the 
sufficent  condition  for  the  almost  sure  asymptotic  stability  of  a  linear  stochastic  system  given 
(Af ,  /3)-stability  in  the  mean. 


4  Stabilizing  Q  and  Transient  Bound  on  Uncertain  System 

The  Lyapunov  analysis  given  in  Theorem  3.5  indicates  a  lower  bound  requirement  on  the  system 
process  noise  covariance,  Q,  and  an  upper  bound  requirement  on  system  measurement  noise 
covariance,  R,  in  order  for  the  controlled  system  to  be  (Af,  /3)-stable  in  the  mean.  Since  the 
measurement  noise  covariance  can  be  obtained  from  sensor  calibration,  the  process  noise  matrix 
Q  is  usually  treated  as  a  tuning  parameter.  This  would  compel  one  to  select  an  extremely 
large  Q  so  that  the  stability  is  always  guaranteed.  Selecting  a  large  Q  value  would  force  the 
estimator  to  completely  rely  upon  the  measurement  signal  and  therefore  the  noise  associated 
with  the  measurement  signal  is  directly  transmitted  into  the  estimates.  This  could  result  in 
a  noisy  control  signal  which  could  lead  to  problems  such  as  chattering.  This  section  shows  a 
systematic  approach  to  select  a  stabilizing  Q  using  the  overbounding  method  of  Petersen  and 
Hollot  (Petersen  and  Hollot  1986,  Douglas  and  Athans  1994). 

Assume  the  structure  of  the  uncertainty  AY(f)  is  given  as 

l 

AT(*)  =  X><(*)Ti  (41) 

i=  1 

where  Yj  is  assumed  to  be  a  rank-one  matrix  of  the  form  Y i  =  tje^.  In  the  above  description, 
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ri(t)  is  the  ?'th  component  of  the  vector  r(t)  €  M1  and  is  upper  bounded  by 

r  >  sup  \ri(t)\,  Mi  <E  {1,  2,  1}  (42) 

t>t0 

Define  matrices  T  and  E  as 

i  i 

T  =  ^2  tjtf  and  E  =  ^  e?:ef  (43) 

i=l  i=  1 

Lemma  4.1:  If  the  uncertain  matrix  AT (t)  has  the  structure  given  in  (41),  then  the  following 
matrix  inequality  is  valid  for  all  matrices  V*  ( t )  of  appropriate  dimensions: 

V*T{t)ATT{t)  +  AT(t)V*{t)  <  f2T  +  P*T{t)EV*{t),  Mt>t0  (44) 

where  f,  T ,  and  E  are  from  (42)  and  (43) . 

Proof  Substituting  T,  =  t into  (41)  yields, 

i 

AT(t)V*(t)  +  V*T(t)ATT(t)  =  ^2  {ri(t)tieJV*(t)  +  n(t)V*T (t^tf] 

i= 1 


Notice 


ri(t)ti 


V*T(t)ei 


n(t)ti 


>  o 


Thus 

rf(t)titj  +  V*T {t)eief V* (t)  >  r;(t)t?:ef V* (t)  +  n(t)V*T (t)eitj 

and 


i  L 

J2  {riWitf  +  V*T(t)eiefP*(t )}  >  ^  {ri(t)tiefv*  (t)  +  ri(t)7?*T(t)eitf  } 


1=1 


i=  1 


Now  substituting  for  T  and  E  yields 

r2T  +  V*T{t)EV*{t)  >  AT(t)V*(t) +V*T(t)ATT(t) 


□ 


A  computationally  feasible  procedure  for  the  calculation  of  a  stabilizing  Q  is  given  next. 

Theorem  4.2:  Assume  the  uncertain  matrix  AY(f)  has  the  structure  given  in  (41)  and  the 
process  noise  covariance,  Q* ,  is  selected  so  that  the  following  matrix  differential  Riccati  equation 
has  a  bounded  positive  definite  matrix  solution,  P*(t): 

V*(t )  =  T  (t)P*(t)  +  P*(t)fT{t)  -  7  P*{t)P*{t)  +  n  (i Q *,  R-1)  (45) 


and 


IZ{Q*,R~l)  >7 V*(t)V*(t)  +  r2T  +  V*T{t)EV*{t),  Mt>t0 


(46) 
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where  7  is  a  positive  constant  and  1Z  [Q* ,  R^1)  denotes  a  positive  definite  matrix  function.  Then, 
the  uncertain  system  in  (30)  is  (M,  (3) -stable  in  the  mean  and 

Mg  <  sup  crmax  (P*(t))  / amin  (P*(t0))  (47) 

t>t0 


where  Mg  represents  the  transient  bound  of  the  uncertain  system’s  mean  response. 

Proof  Since  T (t)  is  assumed  to  generate  an  exponentially  stable  evolution  operator,  there  exists 
a  bounded  positive  definite  matrix,  P*(t),  that  satisfies  equation  (45).  Note  that  (45)  can  be 
written  as 

p*(t)  =  [T(t)  +  AT (t)]  P*{t)  +  P*{t)  [T(t)  +  AT (t)]T  -  7 P*{t)P*{t)  + 1Z  (Q*,  RT1) 

-  AT (t)P*(t)  -  P*{t)ATT(t) 


The  solution  to  above  equation  is 


P*(t )  =  4>(t,  t0)P*(t0)$T(t,  t0)  +  j*  $(t,  r){77  (Q*,  R-1)  -  7 P*(t)P*(t)- 

AT  {t)P*(t)  -  r(r)ATT(r)}$T(t,  r)dr 


where  4>(t,to)  is  the  evolution  operator  generated  by  T (t)  =  T(t)  +  AT (t).  Based  on  Lemma  4.1 
and  the  matrix  inequality  equation  (46) 

U  (Q*,  R-1)  -  7 P*(t)P*(t)  -  AT (t)P*(t)  -  P*(t)ATT(t.)  >  0 


Thus 


P*(t)  >  ^(t,t0)P*(t0)^T(t,t0)  >  <7min  (P*(to))  $(t,t0)$T(t,t0) 


Now  (47)  follows  from 


^max  (P*(t))/ 

^min  (P*(t 0))  >||  $(t,t 0)  ||2 

Therefore,  4>(t,io)  generates  an  exponentially  stable  evolution.  □ 

Assuming  the  system  uncertainties  can  be  written  in  the  form  given  in  (41),  a  stabilizing 
process  noise  covariance,  Q*,  can  be  calculated.  Notice  that  bounds  on  the  system  uncertainties 
used  here  may  be  highly  conservative  and  therefore  it  may  result  in  an  extremely  large  value  of 
Q.  As  mentioned  earlier,  selecting  a  large  Q  results  in  a  noisy  control  signal  and  it  could  lead  to 
problems  such  as  chattering.  Also  note  that  obtaining  the  upper  bound  f  is  rather  difficult  since 
the  system  uncertainties,  A F(t)  and  A D(t),  may  depend  on  the  estimator  gain,  K(t).  Thus 
increasing  the  process  noise  covariance  would  also  increase  the  upper  bound  on  the  uncertainty, 
i.e.,  f.  Finally,  the  reader  should  realize  that  the  dependency  of  system  uncertainties  on  the 
estimator  gain  is  eliminated  if  the  control  distribution  matrix  is  precisely  known,  i.e.,  A B  =  0. 
For  more  details,  please  refer  to  (18). 
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5  Adaptive  Scheme 

After  substituting  the  disturbance  accommodating  control  law,  u(i)  =  SZ(t),  the  plant  dynamics 
in  (1)  can  written  as 


X(t)  =  AX(t)  +  B  I  SZ(t)  +  B_1W(i) 


and  the  estimator  dynamics  in  (14)  can  be  written  as 


(48) 


kt)  =  [Fm  +  DmS]  Z (t)  +  (49) 

where  Y (i)  =  [Y(t)-Y{t)\.  Let  Xext(i)  =  [Xf  (t)  X%(t)  W T(t)f,  now  based  on  assumption  3.1, 
the  controlled  plant  in  (48)  can  be  written  as 


Xext(f)  =  AxtXext(f)  +  BextSZ(t)  +  GV(t)  (50) 

where 


'Ai 

A2 

0(n— r)  xr 

0(n— r)  xr 

^ext  — 

As 

^44 

Irxr 

and  Bext  = 

B 

A 

_nWi 

A 

uw2 

A 

nw3 

0  rxr 

The  following  assumptions  are  now  made. 

Assumption  5.1 :  The  pair  (-4ext> l^ext)  is  controllable  and  the  pair  (*4ext,iL)  is  observable. 

Assumption  5.2:  There  exist  an  r  x  m  matrix  II  such  that  nrII  >  Imxm ,  he.,  m  <  r.  If 
rn  >  r,  then  the  r-outputs  considered  here  are  selected  such  that  the  corresponding  (Aext,  H)  is 
observable. 

Assumption  5.3 :  There  exists  an  m  x  rn  matrix  R  >  0  such  that  Vi  >  to,  we  have 


E 


Y(t)YT(t) 


>  R 


Based  on  assumptions  5.1,  5.2,  and  5.3,  an  adaptive  scheme  for  selecting  the  stabilizing  process 
noise  covariance  matrix  can  be  developed  as  shown  next. 

Theorem  5.4:  Given  assumptions  5.1,  5.2,  and  5.3,  the  controlled  system  is  mean  square 
stable,  E[X(t)]  €  L2  n  L and  X(t)  is  asymptotically  stable  in  the  first  moment,  i.e., 

lim  E  [X(t)]  =  0 

t — ^OO 

if  the  process  noise  covariance  is  updated  online  using  the  adaptive  law 

dQ(t)  =  { AQQ(t)  +  Q{t)Aq  +  7nY(i)YT(i)nT}  dt  (51) 

where  Aq  is  anrxr  negative  definite  matrix  such  that  0  <  — 2Tr  {Aq}  <  1  and  7  is  the  adaptive 
gain. 

Proof  of  this  theorem  is  based  on  the  following  lemmas. 

Lemma  5.5:  Consider  the  following  linear  stochastic  system 


Z  (t)  =  AZ(t)  +  U(t) 
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If  the  matrix  A  generates  an  exponentially  stable  evolution  operator  <5>A(t  —  to)  and  U(t)  €  L2, 
i.e., 


E 


1/2 


|U(r)|2cir 


<  00 


where  \  ■  \  represents  the  Euclidean  norm,  then  Z(t )  G  L2  D  and 


lim  E[Z(t)]  =  0 

t—>  OO 


Proof  The  solution  Z (t)  can  be  written  as 


Z(t) 


$A(t-t0)Z(t0)+  $A(t  -  r)U(r)dr 


'to 


Since  &A(t  —  to)  is  exponentially  stable 


II  ^A(t-to)  ||  <  \oe-a{t~to)  <  Ao,  Vt  >  t0 

where  ||  •  ||  represents  any  induced  matrix  norm  and  Ao  and  a  are  two  positive  constants.  Thus 

|Z(t)|  <||  $A(t  -  to)  II  |Z(to)|  +  /  II  $A(t  -  T )  II  |U(r)|dr 

J  to 

<  A0e-a(t-to)|Z(t0)|  +  rA0e-a(t-r)|U(r)|dr 

J  to 

<  A0e-a(t_t°)|Z(t0)|+  [  \0e~<'a~a°^2^t~T'> e~ao/^(t-r)  |u(r)|dr 

Jto 

The  last  inequality  is  obtained  by  expressing  e~ad~T')  as  e-(a-ao/2)(t-T)e-a0/2(t-r)^  wjiere  aQ  <  2a 
is  a  positive  constant.  Applying  the  Schwartz  inequality  yields 

/  ft  \  V2  /  cf  \  i/2 

|Z(t)|  <Aoe-^-^|Z(to)|+Ao  (jf  e-(2°-«o  )(t-r)drJ  g-ao  (t-r)  |u(r)|2drj 


Thus 


|Z(i)l  <  +  A°  ( 

V  (2a  -  a0)  v. 


rt  \  I/2 

I  e-aoh-r) |u(r)|2(tr 

to 


and 


E  [|Z(t)|]  <  Xoe-^-^E  [|Z(t0)|]  + 


y7 (2a  -  ao) 


,E 


rt  \  I/2' 

e-ao(t-r)|U(r)|2dr\ 


'^0 


/ 


lim  E  [Z(t)|  =  0 


Therefore 
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Also  note  that  £7[|Z(t)  |]  is  bounded  by 


E[\Z(t)\}<\0e-a^E[\Z(t0)\]  + 


y/ (2a  -  a0) 


E 


a |u<T>i2,iT) 7 


Since  U(i)  G  L2 


E 


to 


|U(r)|2drJ 


\  1/2' 


<  E 


|U(r)|2dr 


't0 


1/2 


<  00 


Note  that  Z  (t)  G  L2  since 


o—a0(t—r) 


J  to  to 

Finally  note  that 


)/*oo  /  POO  \  1  POO 

dt<  |U(r)|2  /  e-ao(t~T)dt )  dr  <  —  /  |U(r)|2dr 

Jt0  \Jt0  J  ao  Jt0 


Z(t)€L2=>E[Z(t)]  GL2 


Lemma  5.6:  Consider  the  following  linear  stochastic  system 

Z  (t)  =  AZ(t )  +  U  (i) 

Y(t)  =  C'Z(t) 

//  (A,  C)  is  observable,  Y (t)  G  L2  and  U(t)  G  L2,  then  Z(t)  G  L2  fl  and 

lim  E  [Z(t)\  =  0 


□ 


Proof  If  (A,  C )  is  observable,  then  there  exist  a  matrix  K  such  that  A0  =  A—KC  is  exponentially 
stable.  Now  Z(t)  can  be  written  as 

Z  (t)  =  A0Z(t)  +  U  (t)  +  KY  (t) 

Thus  from  Lemma.  5.5  one  could  conclude  that  Z(t)  G  L2  (~l  L ^  and 

lim  E  [Z  (t)\  =  0 

t—¥  OO 


□ 

The  stability  analysis  given  in  section  3  reveals  that  selecting  a  sufficiently  large  process 
noise  covariance  would  guarantee  asymptotic  stability  of  the  controlled  system’s  mean  response. 
Thus  the  adaptive  law  given  in  Theorem  5.4  increases  the  process  noise  covariance  to  ensure 

that  E  [y(£) 


G  L2.  Now  based  on  the  above  lemmas  the  proof  of  Theorem  5.4  can  be  easily 


obtained  as  shown  next. 

Proof 

Let  denotes  a  filtration  generated  by  Y (t),  i.e. 


E 


Y  (s)  I  E? 


=  Y(s) 


s  <  t 
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Now  consider  the  following  nonnegative  function: 


r  ~  -1 

T 

r 

r  ~  "I 

T  r 

V(to,t,Z,Y,P,Q)=  /  E 
Jto 

Y(r)  |  7£ 

E 

Y(r)  |  7£ 

dr  +  E 

Z(t)  1  ^ 

yf; 

+Tr{Q*  -  Q(t) |  +  Tr|p*ax  -  P{t) | 


where  Q*  >  Q{t)  Vt  >  to  is  a  stabilizing  process  noise  covariance  and  Pr*ax  is  selected  such  that 
Pj*ax  7  P*(t)i  Vt  >  t0,  where  P*(t)  may  be  obtained  by  solving  the  continuous-time  matrix 
differential  Riccati  equation: 


P*(t)  =  FmP*(t)  +  P*(t)FmT  -  P*{t)HTR~lHP*it)  +  GQ*Gt,  P*{t0 )  =  P0 


Note  that  for  any  Q[t)  <  Q*.  P{t)  <  P*iax,  where  P(t )  satisfies 

P(t)  =  FmP(t )  +  P(t)FmT  -  P{t)HTRTlHP{t)  +  GQ(t)GT,  P(t0 )  =  P0  (52) 


More  details  on  this  can  be  found  in  the  comparison  results  given  in  chapter  4  of  Abou-Kandil 
et  al.  (2003).  The  matrix  if  is  a  positive  definite  matrix  of  appropriate  dimensions  and  it  is 
selected  so  that  it  satisfies  the  following  matrix  inequality 


X  [Fm  +  DmS }  +  [Fm  +  DmS]T  X  +  XX +  M<  0 


where  M  >  0.  It  is  important  to  note  that  the  expectation  given  in  the  above  nonnegative 
function  is  conditioned  on  the  filtration  at  the  lower  time  limit.  For  example,  consider  a  time 
instant  s  such  that  to  <  s  <  t,  now  V ( s ,  t ,  Z,  Y,  P ,  Q )  can  be  written  as 


fE 

J  s 

*<1 

T 

F? 

dr  +  E 

Z  it)  |  7f 

T  r 

yf; 

m  i  Ps 

+Tr<j  Q*  -  Q(fH  +  Tr<Xax  -  P(t) 


Now  dV ( s ,  t,  Z,  Y,  P ,  Q )  can  be  calculated  as 


l  T 


dV(s,t,Z,Y,P,Q)  =E  Y [t)  |  7/  E  Y{t)  |  7) 


dt  +  E 


dzit)  |  jQ 


l  T 


XE 


Z(t)  |  7? 


+ 


E 


m  i  t-; 


yf; 


dZ(t)  I  T’7  -  Tr<^  dQ(t)  ^  -  Tr<^  cFP(t) 


Note  that 


dZ(t)  |  7f  =  [Fm  +  DmS }  E  Z  it)  |  7) 


rY 

s 


dt  +  E 


l<it)Yil)  |  7) 


dt 
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Thus 

dV(i 

5,  t.  Z, 

, 

II 

s 

eg 

& 

|jf]T£[Y(t)| 

xf  dt-  Trj 

|  -Tr|dP(t)|  + 

E 

Z(t) 

|  Xff  [Fm  +  DmS] 

T  XE  \z(t)  j  Xf 

dt  +  E\K(t)Y(t)  | 

Xf]7  XE  \z,(t)\  jf]  dt+ 

E 

Z(t) 

|  Xf  X  [Fm  +  Dm 

S\E[±(t)\Xf 

dt  +  E  Z  (t)  | 

T 

xY 

u  s 

XE  \K(t)Y(t)  |  Xf]  dt 

Notice  that  for  any  two  vectors  a  and  b  of  same  dimensions,  the  following  inequality  holds 

aTa  +  bTb  >  arb  +  b7  a 


E  [z (t)  |  XfY  XXE  [Z (t)  |  Xf  ]  +E\K(t)Y{t)  \  XfV  E  \lC(t)Y(t)  \  Xf]  > 

E  \K(t)Y{t)  |  XfY  XE  [z (t)  |  Xf]  +E\z(t)  \  XfY  XE  [if(t)Y(t)  |  Xf 

Therefore 

dV  (s,  t,  Z,  Y,  P,Q)  <  E  Y(t)\EjTE  Y(t)\Xf  dt  -  TrjdQ(t)j  -  TrjdP(t)j  + 

E  [z (t)  |  Xf  ]T  lx  [Fm  +  DmS }  +  [Fm  +  DmS]T  X  +  XX^E  [z (t)  \  Xf]  dt+ 

E  \K(t)Y(t)  |  xf] 1  E  \K(t)Y(t)  |  Xf]  dt 
Now  employing  the  Cauchy-Schwarz’s  inequality  gives 

dV(s,t,Z,Y,P,Q)  <  E  YT(t)Y(t)\Xf  dt  +  E  YT(t)KT(t)K(t)Y(t)  \  Xf  dt  -  Tr  jdQ(i) 
+  E  [Z  (t)  |  Xf]T  [Fm  +  DmS]  +  [Fm  +  DmS]T  X  +  XX^E  ]±  (t)  \  Xf]  dt  -  Tr  jdP(f)  j 
Substituting  (51)  and  (52)  yields 

dV(s,t,Z,Y,P,Q)  <  E  |~YT(t)Y(t)  \  xf  1  dt  +  E  \YT(t)KT(t)K(t)Y(t)  \  Xf]  dt 

-  E  Z (t)  |  Xf  T  ME  Z (t)  |  Xf  dt  -  TV  ^AQQ{t)  +  Q{t)Aq  +  7nY(t)YT(t)nT}  dt 

-  Tr  { FmP{t )  +  P(t)FmT  -  P{t)HTR~lHP{t )  +  GQ{t)GT }  dt 
Note  that 


-Tr  {AQQ{t)  +  Q{t)ATQ}  =  -2Tv  {AQQ{t)}  <  -2Tr  {AQ}  Tv  {Q(t)}  <  Tv{Q(t)} 

The  first  inequality  is  valid  because  —  Aq  is  positive  definite  and  the  process  noise  covariance 
Q{t)  is  positive  semi-definite  (Yang  2000).  The  last  inequality  holds  since  0  <  — 2Tr  {Aq}  <  1. 
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Also  note  that  due  to  the  nature  of  matrix  G,  we  have 

Tr  {GQ{t)GT}  =Tr{Q(f)} 

Thus 

dV(s,t,Z,Y,P,Q)  <  Tr  {e  Y(t)YT(t)\Ej  jdt  +  Tr^E  KT{t)K(t)Y(t)YT(t)  \  Ej  }  dt 

-  E  Z  (t)  |  Ej  1  ME  Z  (t)  Ej  dt-  7Tr  {  Y(t)YT(f)IITn}  dt 
+  Tr  {P{t)HTRElHP{t)}  dt  -  2Tr  { FmP(t )}  dt 
<  E  (1+  ||  K{t)KT{t)  ||)  Tr  |  Y (t)YT(t)|  |  Ej  dt  -  E  Z(t)  \  Ej  T  ME  Z (t)  |  Ej  dt 
-  7Tr  | Y (i)YT(i)  j  dt  +  Tr  {P{t)HT R^HPR)}  dt  -  2Tr  {FmP(t)}dt 

The  second  inequality  holds  since 

|Y(f)|2  +  |ii  (t)Y(f)|2  <  (1+  ||  K(t)KT(t)  ||)  |Y(t)|2 

Therefore  E  dV ( s ,  t,  Z,  Y,  P,  Q )  |  E (X  can  be  written  as 

E  \dV(s,t,Z,Y,P,Q)  |  Ej 1  <  E  I" (1+  ||  K{t)KT{t)  ||)  |Y(t)|2  |  Ej  1  dt 

-E\z{t)  |  MB  [z(t)  |  dt-7£  I" |Y (t)|2  |  Ej 1  dt 

+  E\Tr{P(t)HTR~1HP(t)  -  2 FmP(t)}  \  Ej]  dt 

Combining  the  similar  terms  yields 

E  \dV(s,t,Z,Y,P,Q)  \Ej]<E  [(1+  ||  K(t)KT{t)  ||  -7)  |Y(t)|2  |  Ej]  dt 

-E\z{t)  |  EjV  ME  [z  (t)  |  Ej]  dt  +  eWt  {P(t)HT  RTl  HP{t)  -  2FmP{t)}  |  J7/!  dt 
Let  7  =  7i  +  72  where  7i  is  selected  such  that 

7i  >  1+  II  K(t)KT(t )  ||  (53) 

Thus 

E  dV(s,t,Z,Y,P,Q)  |  Ej  <-72Tr{£  Y(t)YT(t)  \  Ej  }  dt- 

E  \z (t)  |  Ej] 7  ME  \z(t)  |  Ej]  dt  +  E  [Tr  {P{t)HT  R~lHP(t)  -  2 FmP(t)}  |  J7/]  dt 
Now  based  on  assumption  5.3,  we  have 

E  \dV(s,t,Z,Y,P,Q)  |  Ej]  <  -E  \z (t)  \  Ej V  ME  \z{t)  \  Ej]  dt+ 

E  [Tr  { P{t)HTR~lHP{t )  -  2 FmP(t)}  -  72Tr  {R}  \  Ej]  dt 
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Finally  note  that  E  dV (s,  t,  Z,  Y,  P,  Q )  |  P, 


<  0  if 


72  >  Tr  {P(t)HTR~1HP(t)  -  2 FmP(t)}  Tr  {P} 


-l 


(54) 


Assuming  Y (to)  is  precisely  known  yields, 


E 


Y(t)  \F?  =E  Y (t) 


Thus  selecting  71  and  72  according  to  Eqs.  (53)  and  (54)  yields 


E 


V(to,t,Z,Y,P,Q)\-V(t0,t0,Z,Y,P,Q)=  /  E  \ dV (to, r,Z,Y,P,Q) 

J  to 


<  0 


Therefore 


E 


V(t0,  t,  Z,  Y,  P.  Q)  <  zT(t0)  Yz(to)  +  Tr  Q*  -  Q(t0)  +  Tr  P*ax  -  P(t0) 


Also  note 


E 


V(s,t,Z:Y,PQ)\Ej  -V(t0,s,Z,Y,P,Q)=  E \dV(s,T,Z, Y,P,Q)  \  Pt 


<  0 


Thus 


E 


V(s,t,Z,Y,P,Q)\Pj  <V(to,s,Z,Y,P,Q) 


Now  the  properties  of  V (to,t,  Z,  Y,  P ,  Q)  may  be  summarized  as 
(*)  V(t0,t,Z,Y,P,Q)  >0 


(ii)  E 

V(to,t,Z,Y,P,Q) 

<  00 

(in)  E 

V(to,t,Z,Y,P,Q) 

1  PY] 

\  J  S 

<V(t0,s,Z,Y,P,Q), 

s  <  t 

(iv)  V (to,  t.  Z,  Y,  P,  Q)  is  adapted  to 

These  properties  imply  that  V(to,  t,  Z,  Y,  P,  Q)  is  a  nonnegative  Ej -supermartingale  (Kushner 
1967,  Liptser  and  Shiryayev  1989)  and  the  nonnegative  supermartingale  probability  inequality 
yields  (Doob  1953) 


PI  sup  V (t0,  t,  Z,  Y,  P,  Q)  >  A)  < 

H>t0 


zT(t0)Yz(t0)  +  Tr  Q*  -  Q(t0)  +  Tr  P*ax  -  P(t0) 


A 


where  A  >  0  is  any  positive  constant.  Thus  selecting  sufficiently  large  A  yields 


PI  sup  Y(to,t,  Z,  Y,  P,  Q)  <  00)  =  1 

H>t0 


December  23,  2010 


11:32 


International  Journal  of  Control 


IJC'ADAC'MainFile 


24  J.  George  et  al. 

That  is,  V(to,t,Z,Y,P,Q)  is  almost  surely  bounded.  Note  V(to,  t,  Z,  Y,  P,  Q)  is  defined  as 


r  -| 

T 

r  -| 

r  n 

T 

r  -| 

V(t0,t,Z,Y,P,Q)  =  /  E 
J  to 

Y(r) 

E 

Y(r) 

dr  +  E 

m 

XE 

m 

+  TrjQ*  -  Q(t)j  +  Tr{p*ax  -  P(t) } 


Therefore 


V (t0,  t,  Z,  Y,  P,  Q)  £  a.s. 


E 


Y  (t)  £  L2,  Q(t )  £  and  P(t )  €  Lc 


a.s. 


Since  P(t)  is  a.s.  bounded,  the  estimator  gain,  K(t )  =  P(t)HTR  1,  is  also  a.s.  bounded.  Thus 
there  exist  a  k*  such  that 


sup 

t>t0 


K(t)  \\>k*  =0 


The  estimator  dynamics  is  given  as 


i(t)  =  [Fm  +  DmS]  Z (t)  +  K(t)Y(t) 


Since  [Fm  +  DmS ]  generates  an  exponentially  stable  evolution  operator,  and  since  E[Y(t)]  £  L2, 
based  on  Lemma  5.5,  it  can  be  shown  that  E[Z(t)}  £  L2  (~l  Loo,  and 


lim  E 

t—>  OO 


Z(t) 


=  0 


Since  E\Y(t)]  £  L2, 


E[Z(t)\  £L2=>  E[Y(t)}  £  L2 

Now  given  the  observability  assumption,  based  on  Lemma  5.6,  it  can  be  shown  that  i£[X(t)]  £ 
L2  n  Loo,  and 


lim  E  [X(t)]  =  0 

£—>•00 

Finally  note  that  the  controlled  closed-loop  system  can  be  written  as 
Xext(i) 

kt) 


^4ext  ^ext  S 

Xext  (t) 

+ 

-  GV(t)  ' 

K(t)H  {Fm  +  DmS  -  K(t)H}_ 

Z(t) 

K(t)Y(t)_ 

Note  that  the  closed-loop  state  matrix 


Fch(t) 


A^C'Xt  s 

I<(t)H  {Fm  +  DmS  -  K(t)H} 


(55) 


is  bounded.  Also,  the  asymptotic  stability  of  £,[Xext(0]  and  F[Z(t)\  implies  that  the  matrix, 
Fch(t),  generates  an  asymptotically  stable  evolution  operator,  &ch(t,to),  i.e., 

lim  ||$cl(Mo)||  =  0 
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Equation  (55)  can  be  written  in  Ito  form  as 

deceit)  =  FCL(t)XCL(t)dt  +  TcL(t)dBcL(t)  (56) 

where 

rcL (t)  =  ^  and  E  [d&cL(t)dB^L(t)\  =  ^  ^  dt  =  QChdt 

Remark  5  :  It  is  important  to  note  that  if  one  wishes  to  express  (55)  in  Stratonovich  form,  the 
results  given  here  holds  since  we  are  considering  linear  stochastic  differential  equations  with  state 
free  diffusion  term  and  the  solution  obtained  from  the  Stratonovich  integral  equation  converges 
a.s.  and  uniformly  to  that  obtained  from  the  Ito  integral  equation.  For  more  details  please  refer 
to  the  Wong-Zakai  theorem  (Grigoriu  2002). 

Now  using  Ito  formula  d  (XcL,(i)XQL(i))  can  be  written  as 

d  (Xcl^XclW)  =  XCL {t)d  (XCL(i))T  +  d  (XCL(*))  x£L(t)  +  TcUt)QcLrCL(t)dt 

=  {xcl  (t)x£L(t)i£L(t)  +  FcL(t)XcL(t)x^L(t)  +  rCL(t)QCLrCL(*)}<ft 
+  r  CL(t)dBCL(t)X£L(t)  +  XCL(t)dBlL(t)V^L(t) 

Thus 

XCL(*)X£L(f)  =  <f>CL(t,  to)XCL(to)XgL(to)^L(^  to)  +  r  [XcL(T)[rcL(T)dBcL(T)]rlr 

Jto  1 

+  [  XcL(r)[rCL(r)dBcL(r)]r  +  f  $ch(t,  T)rCL(T)QcL^ch(T)^CL(t^  T)dr 
J  to  J to 

Therefore 

E[xCL(t)x£L(t)]  =  E[d>cL(t,to)XcL(to)XgL(fo)^L(^io) 

+  [  ^Ch(t,T)TCL(T)QcL^Ch(T)^CL{t,T)dT 

j  to 

Since  ihcL(Mo)  is  an  asymptotically  stable  evolution  operator  and  Tcl(^)  is  bounded,  it  can  be 
easily  shown  that  the  closed- loop  system  is  mean  square  stable,  i.e., 

lim  £[XCL(f)x£L(f)]  <  M 

t — ^OO 

where  M  is  a  constant  square  matrix  whose  elements  are  finite  (Soong  1973).  □ 

Even  though  the  initial  process  noise  covariance,  Q(to),  may  not  be  the  stabilizing  Q,  the 
adaptive  law  given  in  (51)  can  be  used  to  update  the  process  noise  covariance  online  so  that  the 
controlled  system  is  asymptotically  stable.  A  schematic  representation  of  the  proposed  adaptive 
controller  is  given  in  figure  2. 

6  Simulation  Results 

For  simulation  purposes,  consider  a  two  degree-of-freedom  helicopter  that  pivots  about  the  pitch 
axis  by  angle  9  and  about  the  yaw  axis  by  angle  ip.  As  shown  in  figure  3,  pitch  is  defined  positive 
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Figure  2.  Adaptive  DAC  Block  Diagram 


when  the  nose  of  the  helicopter  goes  up  and  yaw  is  defined  positive  for  a  counterclockwise 
rotation.  Also  in  figure  3,  there  is  a  thrust  force  Fp  acting  on  the  pitch  axis  that  is  normal  to  the 
plane  of  the  front  propeller  and  a  thrust  force  Fy  acting  on  the  yaw  axis  that  is  normal  to  the 
rear  propeller.  Therefore  a  pitch  torque  is  being  applied  at  a  distance  rp  from  the  pitch  axis  and 
a  yaw  torque  is  applied  at  a  distance  ry  from  the  yaw  axis.  The  gravitational  force,  Fg.  generates 
a  torque  at  the  helicopter  center  of  mass  that  pulls  down  on  the  helicopter  nose.  As  shown  in 
figure  3,  the  center  of  mass  is  a  distance  of  lcm  from  the  pitch  axis  along  the  helicopter  body 
length. 


Figure  3.  Two  Degree  of  Freedom  Helicopter 

After  linearizing  about  9(to)  =  V’(^o)  =  9  (to)  =  ip(to)  =  0,  the  helicopter  equations  of  motion 
can  be  written  as 

( Jeq,p  +  mheiillm )0(t)  =  K ppVmpit )  +  KpyVm,y(t)  -  Bp6(t)  +  Wi(t)  (57a) 

(■ Jeq,y  +  mheiil2crn)'ip (t)  =  KyyVm,y(t )  +  KypVm,p{t )  -  Byip(t )  +  W2{t)  (57b) 

A  detailed  description  of  system  parameters  and  assumed  values  are  given  in  Table  2.  The  system 
states  are  the  pitch  and  yaw  angles  and  their  corresponding  rates,  i.e. ,  d(t),  0(t),  and  ip(t). 

The  control  input  to  the  system  are  the  input  voltages  of  the  pitch  and  yaw  motors,  Vm)P  and 
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Vm,y>  respectively.  The  external  disturbances  are  denoted  as  W\  (t)  and  W2(t).  Let  Xi(t)  = 


Table  2.  Two  Degree-of- Freedom  Helicopter  Model  Parameters 


System 

Parameter 

Description 

Assumed 

Values 

True 

Values 

Unit 

Bp 

Equivalent  viscous  damping  about  pitch  axis 

0.8000 

1 

N/V 

By 

Equivalent  viscous  damping  about  yaw  axis 

0.3180 

-0.3021 

N/V 

Jeq,p 

Total  moment  of  inertia  about  yaw  pivot 

0.0384 

0.0288 

Kg-m 2 

Jeq,y 

Total  moment  of  inertia  about  pitch  pivot 

0.0432 

0.0496 

Kg-m 2 

K 

ivpp 

Trust  torque  constant  acting  on  pitch  axis 
from  pitch  motor/propeller 

0.2041 

0.2552 

N-m/V 

K 

J^py 

Trust  torque  constant  acting  on  pitch  axis 
from  yaw  motor/propeller 

0.0068 

0.0051 

N-m/V 

K 

J^yp 

Trust  torque  constant  acting  on  yaw  axis 
from  pitch  motor/propeller 

0.0219 

0.0252 

N-m/V 

K 

lvyy 

Trust  torque  constant  acting  on  yaw  axis 
from  yaw  motor/propeller 

0.0720 

0.0684 

N-m/V 

TYlheli 

Total  mass  of  the  helicopter 

1.3872 

1.3872 

Kg 

1cm 

Location  of  center-of-mass 

0.1857 

0.1764 

m 

[9(t)  m]'T,  X2(t)  =  [0(t)  mf,  u (t)  =  [Vm,p(t)  Vm,y(t)]T ,  and  W (t)  =  [W4(t)  W2(t)}T .  For 
simulation  purposes,  the  external  disturbance  W (t)  is  selected  to  be 


Wx{t)  =  2.430 (t)  -  1.3-0 (t)  -  W\{t)  +  2 W2{t)  +  Vi (t) 
W2(t)  =  -0340(t)  +  1.92^(t)  +  Wi(t)  -  3 W2(t)  +  V2(t) 


(58) 


and 


Vi  (t) 

v2(t) 


V(t)~^(0,lx  10-2/2X2-5(r)) 


Now  the  state-space  representation  of  the  plant  can  be  written  as 


X1(t)  =  X2(t) 

X2(t)  =  A4x2(t)  +  Bu(t)  +  W  (t) 

W  (t)  =  AW2X2(t )  +  AW3W(t)  +  V(t) 


(59) 


'  2.43  -1.3' 
-0.34  1.92 

A  — 
i  nw3  — 

'-1  2  ' 
1  -3 

,  A4  — 

a4  0 

0  a2 

,  B  = 

b\  b2 

63  64 

where 
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and  the  system  parameters  are  given  as 


—Br 


a  i  = 


bi  = 


b3  = 


( Jeq,p  “I- 


Kt 


pp 


(Jeq,p  T  f^heli^cm) 


K, 


yp 


(Jeq,y  T  ^heli^cm) 


—Bn 


a2  = 


b2  = 


b4  = 


(  J eq.y  ^heli^cn 


K, 


py 


(  Jeq,p  “I-  'm-heli^'cn 


K, 


yy 


(  J eq.y  “I-  'nT'heli^cn 


The  state-space  representation  of  the  assumed  system  model  is 

Xlm(i)  =  X2ra(t) 

X2m(t)  =  A4mX2m(t)  +  Bmu(t) 


where 


A4m 


aim  0 
0  a2m 


= 


bim  b2n 

b3m  b4n 


The  measured  output  equations  are  given  as 


Y  (t)  =  CX(f)  +  V  (t) 

where  X(i)  =  [X^ (t)  X^(t)]T  and  C  =  [hx2  02x2]  -  Note  that  the  disturbance  term,  T>(t)  = 
[Dg(t)  V^(t)]T,  can  be  written  as 


Vg(t)  =  Aa\Q(t)  +  Abiui(t)  +  A  b2u2(t)  +  W\{t) 
T>^(t)  =  A  a2ip(t)  +  Ab3ui(t)  +  A  b4u2(t)  +  W2(t ) 

The  assumed  disturbance  term  dynamics  is  modeled  as 


A.(t)  =  -3PAt>+w4*) 


Let  the  extended  assumed  state  vector  be  Z m(t) 
extended  state-space  equation  can  be  written  as 


X^(t)  Vgjt)  V^Jt) 


.  Now  the  assumed 


z  m(t)  —  FmZ  m(t)  +  Dmu(t )  +  GW(i) 


02x2  hx2  02x2 

02x2 

0jx2 

02x2  A4<n  I2x2 

5  Dm  — 

Bm 

and  G  = 

02x2  02x2  Ax>m_ 

_02x2_ 

-^2x2 

where 
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The  estimator  dynamics  can  be  written  as 


Z (t)  =  FmZ(t)  +  Dmn(t)  +  K(t) [CX(f)  -  HZ(t)}  +  K(t)V(t)  (60) 


where  H  =  [C  02x2]-  The  nominal  controller  is  a  linear  quadratic  regulator  which  minimizes 
the  cost  function 


((X,m(f)  -  xd)TQx(Xm(t)  -  xd)  + 


uT  (t)lZuu(t))  dt 


(61) 


where  x^  =  [6d  0  0],  64  and  are  some  desired  hnal  values  of  6  and  i/’j  respectively,  and 

Qx  and  1ZU  are  two  symmetric  positive  definite  matrices.  The  nominal  control  that  minimizes 
the  above  cost  function  is 


u(f)  =  -Km(Xm(t)  -  xd) 


where  I\m  is  the  feedback  gain.  Now  the  disturbance  accommodating  control  law  can  be  written 
in  terms  of  the  estimated  states  and  the  estimated  disturbance  term  as 


u  (f) 


-K„ 


X(t)  -  xd 

v9(t) 


=  SZ(t)  +  Kmxd 


After  substituting  the  above  control  law,  the  true  extended  system  dynamics  can  be  written  as 


'Xi  (t) 

X2  (t) 

= 

V(t). 

02x2  I 2x2  02x2 

02x2  Aa  Rx2 
AVl(t)  AvJt )  AD3(t ) 


'Xi(t)' 

O2X6 

O 

to 

X 

02x1 

X2(t) 

+ 

BmS 

Z(t)  + 

B  K 

Xrf  + 

02x1 

- 

V(t) 

_Bv  1  (t)_ 

Bv  2 

Wa(t) 

where  A-p^t)  =  A BSK(t),  Ap2(t)  =  A A2A4m  +  AW2  -  AW3AA2,  Ap3(t)  =  A A2  +  AW3, 
BVl(t)  =  {A BS  [Fm  +  DmS  -  K(t)H]  +  A A2BmS  -  AWsABS}  , 
and  Bp2(t)  =  AA-2BmKm  —  AW3ABKm.  Here  A B  =  B  —  Bm  and  AA2  =  A4  —  A4m. 

Table  3.  Nominal  Controller/Estimator  Matrices 


LQR  Weighting  Matrices 

Covariance  Matrices 

Ru  =  10  X  1^x2 

Q  _  500  X  I 2x2  02x2 

X  02x2  100  X  I 2x2 

II  II 

O*  ^ 

-to 

l1  l2  ,R=W~3xI2x2, 

/l'S  <?4 

lO^1  X  I 2x2  02x2  02x2 

02x2  I 2x2  02x2 

02x2  02x2  102  X  I 2x2 

Table  3  shows  the  nominal  controller  and  estimator  matrices.  Since  the  measurement  noise 
covariance,  R,  can  be  obtained  from  sensor  calibration,  the  process  noise  matrix,  Q,  is  treated 
as  a  tuning  parameter.  Based  on  the  weighting  matrices  given  in  Table  3,  the  feedback  gain  is 
calculated  to  be 


Kn 


7.0229  0.8239  1.6691  0.3310 
-0.8239  7.0229  -0.0830  2.4486 
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For  simulation  purposes  the  initial  states  are  selected  to  be  [0q  i/>o  #o  = 

[—45°  0  0  0]T  and  the  desired  states  6 d  and  ipd  are  selected  to  be  45°  and  30°,  respectively. 


Figure  4.  Desired  System  Response 

The  desired  response  given  in  figure  4  is  the  system  response  to  the  nominal  control  when 
there  is  no  model  error  and  external  disturbance.  For  illustrative  purposes,  simulations  are 
conducted  using  the  traditional  disturbance  accommodating  control  as  well  as  the  proposed 
adaptive  disturbance  accommodating  control.  Results  obtained  using  the  traditional  DAC  is 
given  first. 


6.1  DAC  Results 

Figure  5(a)  shows  the  unstable  system  response  obtained  for  the  first  simulation  where  the 
disturbance  term  process  noise  covariance  is  selected  to  be  Q  =  103  x  12x2-  Figure  5(b)  shows  the 
input  corresponding  to  the  first  simulation  scenario.  Figures  6(a)  and  6(b)  contain  the  estimated 


(a)  (b) 

Figure  5.  Actual  States  and  Input:  Q  =  103  X  72x2 

disturbance  term  and  the  error  between  the  desired  states  and  the  true  states  corresponding  to 
the  first  simulation.  Note  that  the  first  simulation  results  given  in  figures  5  and  6  are  unstable 
due  to  the  low  value  of  Q  selected. 

A  second  simulation  is  conducted  using  Q  =  105  x  hx2-  The  system  response  obtained  for 
the  second  simulation  is  given  in  figure  7.  Figure  8  shows  the  estimated  disturbance  term  and 
state  error  obtained  for  the  second  simulation.  Note  that  the  estimated  system  rates,  estimated 
disturbance  term  and  the  control  input  are  highly  noisy  because  of  the  large  Q  selected. 
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(a)  (b) 

Figure  6.  Disturbance  Term  and  State  Error:  Q  =  103  X  72x2 


Figure  7.  Actual  States  and  Input:  Q  =  10s  X  72x2 


Figure  8.  Disturbance  Term  and  State  Error:  Q  =  10s  X  72x2 


The  results  shown  here  indicate  that  for  a  small  value  of  process  noise  covariance,  the  controlled 
system  is  unstable.  Though  a  large  value  of  process  noise  covariance  stabilizes  the  controlled 
system,  it  also  results  in  highly  noisy  estimates.  The  direct  dependency  of  the  controlled  system’s 
stability  on  the  process  noise  covariance  is  more  evident  in  the  simulation  results  given  next. 

Combining  the  plant  dynamics  in  (59)  and  the  estimator  dynamics  in  (60),  the  closed- loop 
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system  dynamics  can  be  written  as 


Xf(f) 

02x2  I 2x2  02x2 

02x6 

'Xf(t)' 

1 - 

X 

CN 

O 

i _ 

X2(t) 

02x2  ^4  I 2x2 

BS 

X2(t) 

+ 

BKrtiX-d 

W(i) 

02x2  7lW2  AW3 

02x6 

W(t) 

V(i) 

_K(t)  02x2  02x2 

(Fm  +  DmS  -  K(t)H)_ 

Z(t) 

pmKmxd  +  K(t)V(t)_ 

(63) 

Since  we  are  considering  a  time-invariant  system  here,  the  Kalman  gain  K{t )  converges  to  its 


Figure  9.  System  Closed-loop  Poles  for  Q  Varying  from  1  X  102  to  1  X  10s 

steady-state  value  fairly  quickly.  Using  the  steady-state  Kalman  gain,  the  stability  of  the  closed- 
loop  system  can  be  easily  be  verified.  Figure  9  shows  the  closed-loop  poles  of  the  system  for 
different  values  of  Q  ranging  from  1  x  102  x  12x2  to  1  x  10'5  x  12x2 ■  Figure  9  indicates  that  the 
controlled  system  is  unstable  for  the  initial  small  values  of  Q  and  the  closed-loop  poles  migrate 
into  the  stable  region  as  Q  increases. 


6.2  Adaptive  DAC  Results 

Results  obtained  by  implementing  the  proposed  adaptive  disturbance  accommodating  scheme 
is  presented  in  this  subsection.  Based  on  the  assumed  system  parameters  and  controller  design 
matrices  given  in  Tables  2  and  3,  the  assumed  state  matrix,  Am,  the  assumed  input  matrix,  Bm , 
and  the  DAC  matrix,  S,  can  be  calculated  as 

0  O' 

0  0 
2.37  0.08  ’ 

0.24  0.79 


0  0 

1.00 

0 

0  0 

0 

1.00 

,  Bm  = 

0  0 

-9.28 

0 

0  0 

0 

-3.50 

-7.02  -0.82  -1.67  -0.33  -0.42  0.04 
0.82  -7.02  0.08  -2.45  0.13  -1.28 


and 
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As  shown  in  section  6,  matrix  Ax>m  is  given  as 


Now  the  matrix  [Fm  +  DmS]  can  be  calculated  as 

0  0  1.00  0  0  O' 

0  0  0  1.00  0  0 

f  1  _  -16.56  -2.50  -13.22  -0.98  0  0 

+  _x  04  _5  756  _0  34  _5  51  _Q  Q 

0  0  0  0  -1.00  0 

0  0  0  0  0  -3.00 

Let  M.  =  10_1  x  Iq,  now  the  positive  definite  symmetric  matrix  X  that  satisfies  the  following 
matrix  inequality 

X  [Fm  +  DmS]  +  [Fm  +  DmS]T  X  +  XX  +  M<  0 

can  be  calculated  as 

'  0.1117  0.0005  0.0035  -0.0011  -0.0000  -0.0000' 

0.0005  0.1165  -0.0008  0.0102  -0.0000  -0.0000 

_  0.0035  -0.0008  0.0041  -0.0005  -0.0000  -0.0000 

“  -0.0011  0.0102  -0.0005  0.0110  0.0000  -0.0000 

-0.0000  -0.0000  -0.0000  0.0000  0.0513  -0.0000 

-0.0000  -0.0000  -0.0000  -0.0000  -0.0000  0.0167 

Since  the  number  of  inputs  and  the  number  of  outputs  are  the  same  here,  the  matrix  II  is 
selected  as  the  identity  matrix  12x2 ■  For  the  implementation  of  the  adaptive  law,  the  following 
parameters  are  selected: 

Aq  =  -0.25x/2x2  and  7=  (j|  K(t)  ||  +Tr  {P{t)HT R-1  HP{t)  -  2FmP{t)}  Tr  {i?}”1  +  103) 

Three  different  simulation  scenarios  are  considered  here. 

6.2.1  Simulation  I 

For  the  first  simulation  the  initial  process  noise  covariance  is  selected  to  be  Q(to)  =  10-5  x 
I 2x2-  Figures  10(a)  and  10(b)  show  the  system  response  and  the  disturbance  accommodating 
control  input  obtained  for  the  first  simulation.  Figures  11(a)  and  11(b)  contain  the  estimated 
disturbance  term  and  the  error  between  the  desired  states  and  the  true  states  corresponding  to 
the  first  simulation.  Note  that  the  first  simulation  results  given  in  Figs.  10  and  11  indicate  that 
the  adaptive  scheme  is  able  to  stabilize  and  recover  the  desired  performance  despite  the  initial 
unstable  process  noise  covariance  selected.  The  time  varying  process  noise  covariance  obtained 
for  the  first  simulation  is  given  in  figure  12. 

6.2.2  Simulation  II 

For  the  second  simulation  the  initial  process  noise  covariance  is  selected  to  be  Q(to)  =  hx2- 
Figures  13(a)  and  13(b)  show  the  system  response  and  the  disturbance  accommodating  control 
input  obtained  for  the  second  simulation.  Figures  14(a)  and  14(b)  contain  the  estimated  dis¬ 
turbance  term  and  the  error  between  the  desired  states  and  the  true  states  corresponding  to  the 
second  simulation.  The  simulation  results  given  in  Figs.  13  and  14  indicate  that  the  adaptive 
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(a) 


Figure  10.  Actual  States  and  Input:  Q(to)  =  10  5  X  72  x2 


Figure  11.  Disturbance  Term  and  State  Error:  Q(to)  =  10  5  X  72  x2 


Figure  12.  Adaptive  Process  Noise  Covariance:  Q(to)  =  10  5  X  72X2 


scheme  is  able  to  stabilize  and  recover  the  desired  performance  despite  the  initial  unstable  pro¬ 
cess  noise  covariance  selected.  The  time  varying  process  noise  covariance  obtained  for  the  second 
simulation  is  given  in  figure  15. 
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Figure  13.  Actual  States  and  Input:  Q(£q)  =  ^2x2 


Figure  14.  Disturbance  Term  and  State  Error:  Q(£q)  =  -^2x2 


Figure  15.  Adaptive  Process  Noise  Covariance:  Q(to)  =  /2x2 


6.2.3  Simulation  III 

For  the  third  simulation  the  initial  process  noise  covariance  is  selected  to  be  Q(to )  =  105  x 
hx2 ■  Figures  16(a)  and  16(b)  show  the  system  response  and  the  disturbance  accommodating 
control  input  obtained  for  the  third  simulation.  Figures  17(a)  and  17(b)  contain  the  estimated 
disturbance  term  and  the  error  between  the  desired  states  and  the  true  states  corresponding  to 
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Figure  16.  Actual  States  and  Input:  Q(to)  =  105  x  12x2 


Figure  17.  Disturbance  Term  and  State  Error:  Q(to)  =  105  x  72x2 

the  third  simulation.  Figure  18  shows  the  time  varying  process  noise  covariance  obtained  for  the 
third  simulation. 


x  104 


Figure  18.  Adaptive  Process  Noise  Covariance:  Q(to)  =  10*  X  72x2 

Figure  19  shows  the  time  varying  process  noise  covariance  obtained  for  the  three  simulations. 
Figure  19  indicates  that,  regardless  of  the  initial  matrix  selected,  the  process  noise  covariance 
settles  down  at  its  steady-state  value,  which  is  around  4  ~  5  x  103  for  the  present  scenario. 
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(a)  Q(to)  =  10  5  x  I2x 2  (b)  Q(t0)  =  1 2x2 


x  io4 


7  Conclusion 

This  paper  presents  the  formulation  of  an  observer-based  stochastic  disturbance  accommodating 
control  approach  for  linear  time-invariant  multi-input  multi-output  systems  which  automatically 
detects  and  minimizes  the  adverse  effects  of  both  model  uncertainties  and  external  disturbances. 
Assuming  all  system  uncertainties  and  external  disturbance  can  be  lumped  in  a  disturbance 
term,  this  control  approach  utilizes  a  Kalman  estimator  in  the  feedback  loop  for  simultaneously 
estimating  the  system  states  and  the  disturbance  term  from  measurements.  The  estimated  states 
are  then  used  to  develop  a  nominal  control  law  while  the  estimated  disturbance  term  is  used 
to  make  necessary  corrections  to  the  nominal  control  input  to  minimize  the  effect  of  system 
uncertainties  and  the  external  disturbances.  The  stochastic  stability  analysis  conducted  on  the 
controlled  system  reveals  a  lower  bound  requirement  on  the  estimator  design  parameters,  such 
as  the  process  noise  covariance  matrix  and  the  measurement  noise  covariance  matrix,  in  order  to 
ensure  the  controlled  system  stability.  Since  the  measurement  noise  covariance  can  be  obtained 
from  sensor  calibration,  the  process  noise  matrix  is  treated  as  a  tuning  parameter.  Based  on  the 
stochastic  Lyapunov  analysis,  an  adaptive  law  is  developed  for  updating  the  selected  process 
noise  covariance  online  so  that  the  controlled  system  is  stable.  The  adaptive  scheme  introduced 
here  guarantees  asymptotic  stability  in  the  mean  and  the  mean  square  stability  of  the  controlled 
system.  The  simulation  results  given  here  explicitly  reveal  the  direct  dependency  of  the  proposed 
control  scheme  on  the  process  noise  covariance  matrix.  Since  the  nominal  control  action  on  the 
true  plant  is  unstable,  selecting  a  very  low  process  noise  covariance  resulted  in  an  unstable 
system.  On  the  other  hand,  selecting  a  large  value  stabilized  the  system  but  resulted  in  a  highly 
noisy  control  input.  The  numerical  simulations  indicate  that  the  adaptive  scheme  is  able  to 
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stabilize  and  recover  the  desired  performance  despite  selecting  an  initial  unstable  process  noise 
covariance.  The  results  also  indicate  that  regardless  of  the  initial  matrix  selected,  the  process 
noise  covariance  settles  down  to  its  steady-state  value. 
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